In recent years, there has been an enormous expansion in the volume of data due to the proliferation of technologies such as social networks, smart meters and sensors, cloud computing and more powerful mobile devices. The term “Big Data” is now widely used to refer to arbitrarily large aggregates of varied, complex and minimally structured data. Analysis of larger data sets can result in information that may not be derivable from smaller sets of data. For example, in a single large set of related data, a larger number of correlations can be found as compared to separate smaller sets with the same total amount of data, where fewer correlations can be found owing to the inherent disjunction of separated data sets. Thus, as the desire to gain enhanced information from Big Data increases, greater design resources are being expended towards developing more rapid data processing.
Complex event processing (CEP), as used herein, refers to techniques by which information from multiple data sources are analyzed to infer meaningful events (such as opportunities or threats), preferably with as little processing delay as possible. Current trends are towards performing CEP on ever-widening event clouds that approach the realm of Big Data. The vast amount of information available about events is often difficult to process with conventional data management, processing and analysis tools. Accordingly, there is a need for complex event processing tools for very large event clouds.